由于面向任务导向的对话系统在我们的生活中越来越受欢迎，提出并探索了更现实的任务。然而，出现了新的实际挑战。例如，由于在现有公共数据集中缺少这种情况，当前对话系统无法在查询数据库时有效处理多个搜索结果。在本文中，我们提出了数据库搜索结果（DSR）歧义，这是一个专注于消除数据库搜索结果的新任务，这通过允许它们从多个选项中选择了多个选项而不是只有一个来增强用户体验。为研究这项任务，我们增强了受到流行的面向任务的对话数据集（Multimoz和SGD），转弯，由（a）通过预定义的语法和（b）为子集收集人类释义的（b）来解析歧义。我们发现，我们的增强对话数据的培训提高了模型处理模糊方案的能力，而不会牺牲未修改的转弯。此外，即使在没有域名数据的情况下，也有助于我们的模型帮助我们的模型提高DSR消歧的性能，表明它可以被学习为普遍对话技能。我们的数据和代码将公开可用。

最近延伸预留下芬特的神经模型的神经模型继续实现新的最新导致对话状态跟踪（DST）基准的联合目标准确性（JGA）。但是，我们调查了他们的稳健性，因为它们在JGA中显示了急剧下降，以便与现实扰动的话语或对话框流动的对话。通过清单（Ribeiro等，2020），我们设计了一个名为CheckDST的度量集合，促进DST模型的比较，通过测试具有增强测试集的众所周知的弱点来促进革命性的全面尺寸。我们使用CheckDST评估最近的DST模型，并认为模型应该更全面地评估，而不是在JGA上追求最先进的JGA，因为更高的JGA不保证更好的整体稳健性。我们发现基于跨度的分类模型是有弹性的，不合适的命名实体，但对语言品种不强大，而那些基于自回归语言模型的人概括为语言变化，但往往会记住命名实体并往往是幻觉。由于它们各自的弱点，两种方法都不适合现实世界部署。我们认为CheckDst是未来研究的一个有用指南，用于开发面向任务的对话模型，体现了各种方法的优势。

Charisma is considered as one's ability to attract and potentially also influence others. Clearly, there can be considerable interest from an artificial intelligence's (AI) perspective to provide it with such skill. Beyond, a plethora of use cases opens up for computational measurement of human charisma, such as for tutoring humans in the acquisition of charisma, mediating human-to-human conversation, or identifying charismatic individuals in big social data. A number of models exist that base charisma on various dimensions, often following the idea that charisma is given if someone could and would help others. Examples include influence (could help) and affability (would help) in scientific studies or power (could help), presence, and warmth (both would help) as a popular concept. Modelling high levels in these dimensions for humanoid robots or virtual agents, seems accomplishable. Beyond, also automatic measurement appears quite feasible with the recent advances in the related fields of Affective Computing and Social Signal Processing. Here, we, thereforem present a blueprint for building machines that can appear charismatic, but also analyse the charisma of others. To this end, we first provide the psychological perspective including different models of charisma and behavioural cues of it. We then switch to conversational charisma in spoken language as an exemplary modality that is essential for human-human and human-computer conversations. The computational perspective then deals with the recognition and generation of charismatic behaviour by AI. This includes an overview of the state of play in the field and the aforementioned blueprint. We then name exemplary use cases of computational charismatic skills before switching to ethical aspects and concluding this overview and perspective on building charisma-enabled AI.

Telling stories is an integral part of human communication which can evoke emotions and influence the affective states of the audience. Automatically modelling emotional trajectories in stories has thus attracted considerable scholarly interest. However, as most existing works have been limited to unsupervised dictionary-based approaches, there is no labelled benchmark for this task. We address this gap by introducing continuous valence and arousal annotations for an existing dataset of children's stories annotated with discrete emotion categories. We collect additional annotations for this data and map the originally categorical labels to the valence and arousal space. Leveraging recent advances in Natural Language Processing, we propose a set of novel Transformer-based methods for predicting valence and arousal signals over the course of written stories. We explore several strategies for fine-tuning a pretrained ELECTRA model and study the benefits of considering a sentence's context when inferring its emotionality. Moreover, we experiment with additional LSTM and Transformer layers. The best configuration achieves a Concordance Correlation Coefficient (CCC) of .7338 for valence and .6302 for arousal on the test set, demonstrating the suitability of our proposed approach. Our code and additional annotations are made available at https://github.com/lc0197/emotion_modelling_stories.

Automatic video captioning aims for a holistic visual scene understanding. It requires a mechanism for capturing temporal context in video frames and the ability to comprehend the actions and associations of objects in a given timeframe. Such a system should additionally learn to abstract video sequences into sensible representations as well as to generate natural written language. While the majority of captioning models focus solely on the visual inputs, little attention has been paid to the audiovisual modality. To tackle this issue, we propose a novel two-fold approach. First, we implement a reward-guided KL Divergence to train a video captioning model which is resilient towards token permutations. Second, we utilise a Bi-Modal Hierarchical Reinforcement Learning (BMHRL) Transformer architecture to capture long-term temporal dependencies of the input data as a foundation for our hierarchical captioning module. Using our BMHRL, we show the suitability of the HRL agent in the generation of content-complete and grammatically sound sentences by achieving $4.91$, $2.23$, and $10.80$ in BLEU3, BLEU4, and METEOR scores, respectively on the ActivityNet Captions dataset. Finally, we make our BMHRL framework and trained models publicly available for users and developers at https://github.com/d-rothen/bmhrl.

One of the major challenges in acoustic modelling of child speech is the rapid changes that occur in the children's articulators as they grow up, their differing growth rates and the subsequent high variability in the same age group. These high acoustic variations along with the scarcity of child speech corpora have impeded the development of a reliable speech recognition system for children. In this paper, a speaker- and age-invariant training approach based on adversarial multi-task learning is proposed. The system consists of one generator shared network that learns to generate speaker- and age-invariant features connected to three discrimination networks, for phoneme, age, and speaker. The generator network is trained to minimize the phoneme-discrimination loss and maximize the speaker- and age-discrimination losses in an adversarial multi-task learning fashion. The generator network is a Time Delay Neural Network (TDNN) architecture while the three discriminators are feed-forward networks. The system was applied to the OGI speech corpora and achieved a 13% reduction in the WER of the ASR.

幽默是人类情感和认知的重要因素。它的自动理解可以促进更自然的人类设备互动和人工智能的人性化。当前的幽默检测方法仅基于分阶段数据，使其不适用于“现实世界”应用程序。我们通过引入新颖的Passau自发足球教练幽默（Passau-SFCH）数据集来解决这种缺陷，包括大约11个小时的录音。在马丁的幽默风格问卷中提出的幽默及其尺寸（情感和方向）的存在，请注释Passau-SFCH数据集。我们进行了一系列实验，采用了经过预定的变压器，卷积神经网络和专家设计的功能。分析了每种模式（文本，音频，视频）的表现，以进行自发幽默识别，并研究了它们的互补性。我们的发现表明，对于对幽默及其情感的自动分析，面部表情是最有希望的，而幽默方向可以通过基于文本的功能进行建模。结果揭示了各种主题之间的差异，突出了幽默用法和风格的个性。此外，我们观察到决策级融合会产生最佳认可结果。最后，我们在https://www.github.com/eihw/passau-sfch上公开代码。可以根据要求获得Passau-SFCH数据集。

我们考虑开放的联合学习（FL）系统，客户可以在FL过程中加入和/或离开系统。鉴于当前客户端数量的差异，在开放系统中不能保证与固定模型的收敛性。取而代之的是，我们求助于一个新的性能指标，该指标称我们的开放式FL系统的稳定性为量，该指标量化了开放系统中学习模型的幅度。在假设本地客户端的功能强烈凸出和平滑的假设下，我们从理论上量化了两种FL算法的稳定性半径，即本地SGD和本地ADAM。我们观察到此半径依赖于几个关键参数，包括功能条件号以及随机梯度的方差。通过对合成和现实世界基准数据集的数值模拟，我们的理论结果得到了进一步验证。

我们介绍了本地重新考虑的任务，该任务通过打开和关闭图像中可见的光源来改变场景的照片。这项新任务与传统的图像重新确定问题不同，因为它引入了检测光源并推断出从它们中散发出的光模式的挑战。我们提出了一种用于本地重新考虑的方法，该方法通过使用另一个模型的合成生成的图像对来训练模型，而无需监督任何新型图像数据集。具体而言，我们从样式空间操纵的gan中收集了配对的训练图像；然后，我们使用这些图像来训练有条件的图像到图像模型。为了基于本地重新测试，我们介绍了Lonoff，这是一个在室内空间中拍摄的306张精确对齐图像的集合，其中灯的不同组合打开了。我们表明，我们的方法显着优于基于GAN倒置的基线方法。最后，我们演示了分别控制不同光源的方法的扩展。我们邀请社区解决这项新的当地重新任务。

未来发生的一些事件对于公司，政府甚至我们的个人生活可能很重要。在建立之前对这些事件的预测有助于有效的决策。我们称此类事件新兴实体。它们尚未发生，在KB中没有有关它们的信息。但是，有些线索存在于不同领域，尤其是在社交媒体上。因此，检索这些类型的实体是可能的。本文提出了一种早期发现新兴实体的方法。我们使用短消息的语义聚类。为了评估提案的绩效，我们设计和利用了绩效评估指标。结果表明，我们提出的方法发现了Twitter趋势并非总是能够有能力的那些新兴实体。